Etiquette-based vehicle having pair mode and smart behavior mode and control systems therefor

ABSTRACT

Provided is a self-powered vehicle, comprising: a mechanical drive system, a set of sensors, and a controller coupled to the mechanical drive system to move the vehicle. The self-powered vehicle can operate in a plurality of modes, including a pair mode and a smart behavior mode. In pair mode the vehicle follows the trajectory of a user and in smart behavior mode the vehicle performs autonomous behavior. The self-powered vehicle operates with hysteresis dynamics, such that the movements of the vehicle are consistent with ergonomic comfort of the user and third-party pedestrian courtesy. The self-powered vehicle can operate with other self-powered vehicles in a convoy.

RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 17,395,949 filed Aug. 6, 2021 and claims the benefit of U.S.provisional application Ser. No. 63/061,897, filed Aug. 6, 2020, andU.S. provisional application Ser. No. 63/155,098, filed Mar. 1, 2021.Each of these applications is hereby incorporated herein by reference inits entirety.

TECHNICAL FIELD

The present invention relates to vehicles and control thereof, and moreparticularly to personal vehicles configured to operate in any of aplurality of modes, such modes including a pair mode and a smartbehavior mode, and control systems therefor.

SUMMARY OF THE EMBODIMENTS

In accordance with one embodiment of the invention, there is provided aself-powered personal vehicle. The vehicle of this embodiment includes amechanical drive system to cause movement of the vehicle; a set ofsensors; and a controller, coupled to the mechanical drive system, tomanage movement of the vehicle. The controller is configured to causethe vehicle to operate (i) in a pair mode with a user wherein thevehicle follows, with hysteresis dynamics, a trajectory of such user, soas to exhibit latencies in following movements of the user in a mannerconsistent with ergonomic comfort of the user and third-party pedestriancourtesy and (ii) in a smart behavior mode, in which the vehicle hasexited from the pair mode and executes an autonomous behavior.

In a related embodiment, the controller is configured to cause thevehicle to operate in the pair mode so that the vehicle occupies aspeed-dependent position behind the user. In a further relatedembodiment, the controller is configured to cause the vehicle to operatein the pair mode so that, in the absence of an obstacle in the way ofthe vehicle, the speed-dependent position is also on a default side ofthe user. Optionally, the controller is configured to cause the vehicleto operate in the pair mode so that, in the presence of an obstacle inthe way of the vehicle, the vehicle performs a tuck in the direction ofa position immediately behind the user. As a further option, the tuckperformed by the vehicle follows a curve. And as a still further option,the curve is a cubic spline curve based on parameters including distanceand angle between the vehicle and the user, velocity of the user,velocity of the obstacle, and distance from the vehicle to the obstacle.

In another related embodiment, the controller is configured to cause thevehicle to operate in the pair mode so that, if the user occupies aposition in a dead zone area defined relative to the vehicle, the personis deemed stationary for purposes of following movements of the user.Optionally, the dead zone is teardrop shaped. In a further relatedembodiment, the controller is configured to cause the vehicle to operatein the pair mode so that, if the user is outside of the dead zone butoccupies a position in a track zone area that surrounds the dead zone,orientation of the vehicle is adjusted so that the user is centeredwithin the dead zone. As a further option, the controller is configuredto cause the vehicle to operate in the pair mode so that, if the userleaves the track zone with a velocity in excess of a threshold, thevehicle begins following the user. Optionally, the threshold ispredetermined and is about 0.6 m/s.

In another related embodiment, the controller is configured to cause thevehicle to operate in a smart behavior mode by which the vehicle engagesin autonomous behavior in passing through a doorway. As a furtheroption, the controller is configured to cause the vehicle to perform adoor-passing sequence of discrete actions including a following approachmoment wherein the vehicle in pair mode follows a user up until themoment when the user reaches a vicinity of the doorway; a decouplingmoment wherein the vehicle decouples from pair mode with the user andinitiates the autonomous behavior if a perceived door openness anglereaches a threshold angle; a smarts moment wherein the vehicle engagesin the autonomous door passing behavior and navigates through thedoorway to a recoupling position; a recoupling moment wherein thevehicle re-enters pair mode with the user; and a following walk-awaymoment wherein the vehicle transitions from a waiting position at therecoupling moment to following position in pair mode.

In another embodiment, the invention provides A plurality ofetiquette-based vehicles, each vehicle comprising:

a mechanical drive system to cause movement of such vehicle;

a set of sensors; and

a controller, coupled to the mechanical drive system, to manage movementof such vehicle;

wherein the controller is configured to cause such vehicle to exhibitconvoy behavior so as to participate in a convoy of the plurality ofetiquette-based vehicles having a leader, the convoy behavior including:getting in line and following in line, wherein each etiquette-basedvehicle, other than any etiquette-based vehicle that is the leader andoperating in a smart behavior mode, (i) enters a pair mode with adistinct object, the object selected from the group consisting of theleader and another vehicle of the plurality of etiquette-based vehiclesand (ii) successively joins the convoy and follows the leader, directlyor indirectly, with hysteresis dynamics, as the leader executestraversal of a trajectory.

Optionally, the convoy behavior further includes the stage of leavingline formation of the convoy, wherein the vehicle exits pair mode andenters a smart behavior mode, in which the vehicle autonomously movesout of the line formation. Alternatively or in addition, in getting inline, the vehicle closest to the leader is caused to enter pair modewith the leader and each vehicle successively farther away from theleader is caused to enter pair mode with the vehicle next closer to theleader. Also alternatively or in addition, the leader is anetiquette-based vehicle that has been trained to travel autonomouslyalong a known path.

As a further option to the embodiment described at the beginning of thisSummary, in pair mode, the vehicle is configured to perform a learnedpath training behavior in which the vehicle experiences and stores:

a starting location where the vehicle is located when the trainingbehavior shall have begun;

the trajectory of the user along which the vehicle shall have followedthe user; and

an ending location where the training behavior shall have been ended;

wherein, after storing such experience, in smart behavior mode, thevehicle autonomously traverses the trajectory of the user, from thestarting location to the ending location. Optionally, the trainingbehavior includes storing by the vehicle of a velocity-versus-positionprofile over the trajectory along which the vehicle shall have followedthe user, and in smart behavior mode the vehicle autonomously traversesthe trajectory using the stored velocity-versus-position profile. As afurther option, the velocity-versus-position profile is stored as a setof position-dependent vectors.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of embodiments will be more readily understood byreference to the following detailed description, taken with reference tothe accompanying drawings, in which:

FIG. 1 is a diagram illustrating, in accordance with an embodiment ofthe present invention, from a top view, relative positions of a user 12and an etiquette-based vehicle, in pair mode, in successivespeed-dependent heel-following positions 13A, 13B, and 13C relative tothe user.

FIG. 2 is a diagram illustrating, from a top view, relative positions ofa user and an etiquette-based vehicle, in accordance with an embodimentof the present invention, in pair mode, first (as in FIG. 1 ) insuccessive speed-dependent heel-following positions 13A, 13B, and 13Crelative to the user 12, and second with an obstacle avoidance tuck (toavoid the pedestrian in position 21) showing the vehicle in successivespeed-dependent positions 23A, 23B, and 23C relative to the user 22.

FIG. 3 is a graphic mapping from our studies of motion of individualsperforming tasks in a relatively stationary position.

FIGS. 4A and 4B are diagrams of the dead zone and track zonerespectively in accordance with embodiments of the present invention.

FIG. 5 is a diagram illustrating, from a top view, relative positions ofa user and an etiquette-based vehicle, in pair mode, in successivepositions, as the vehicle slows down in a following approach moment (asdefined below) in relation to a door, in accordance with embodiments ofthe present invention.

FIG. 6 is a diagram illustrating, from a top view, relative positions ofa user and an etiquette-based vehicle, in pair mode, in the decouplingmoment (as defined below) in relation to a door, in accordance withembodiments of the present invention.

FIG. 7 is a diagram illustrating, from a top view, relative positions ofa user and an etiquette-based vehicle, in the smarts moment (as definedbelow) in relation to a door as the vehicle navigates autonomouslythrough the doorway, in accordance with embodiments of the presentinvention.

FIG. 8 is a diagram illustrating, from a top view, relative positions ofa user and an etiquette-based vehicle in the recoupling moment (asdefined below) in relation to a door, in accordance with an embodimentof the present invention after the vehicle has navigated autonomouslythrough the doorway, when the user closes the door and the vehicle isoriented in the most likely direction of the user's gaze.

FIG. 9 is a diagram illustrating, from a top view, relative positions ofa user and an etiquette-based vehicle, in the following walk-away moment(as defined below), in accordance with an embodiment of the presentinvention, after the recoupling moment and vehicle has re-entered pairmode with the user.

FIGS. 10 and 11 are front and rear views respectively of a smart-followmodule fitted with a set of sensors and a controller configured tomanage movement of an autonomous host self-powered vehicle, inaccordance with another embodiment of the present invention.

FIG. 12 is a diagram illustrating, from a top view, relative positionsof a user and an etiquette-based vehicle, at the start of a trainingbehavior (as defined below), in accordance with an embodiment of thepresent invention, before a user has shown the etiquette-based vehiclethe known path.

FIG. 13 is a diagram illustrating, from a top view, in accordance withan embodiment of the present invention, a known path from a start pointto an end point of an etiquette-based vehicle and a user, in a trainingbehavior (as defined below).

FIG. 14 is a diagram illustrating, from a top view, an etiquette-basedvehicle and a user, at the end of a training behavior, in accordancewith an embodiment of the present invention, wherein the vehicle hasbeen instructed to follow the known path, (as defined below).

FIG. 15 is a diagram illustrating, from a top view, an etiquette-basedvehicle exiting smart behavior mode and entering park mode aftertravelling along a known path, in accordance with an embodiment of thepresent invention.

FIG. 16 is a diagram illustrating, from a top view, in accordance withan embodiment of the present invention, an etiquette-based vehicle insmart behavior mode following a known path (as defined below) and auser, which follows a different path.

FIG. 17 is a diagram illustrating, a group of etiquette-based vehiclesforming a convoy in accordance with an embodiment of the presentinvention.

FIG. 18 is a diagram illustrating, from a top view, in accordance withan embodiment of the present invention, a plurality of etiquette-basedvehicles in convoy behavior (as defined below) following a user.

FIG. 19 is a diagram illustrating, from a top view, in accordance withan embodiment of the present invention, a plurality of etiquette-basedvehicles in convoy behavior (as defined below) following a user withhysteresis dynamics.

FIG. 20 is a diagram illustrating, from a top view, in accordance withan embodiment of the present invention, a plurality of etiquette-basedvehicles as convoy behavior is toggled off and the etiquette-basedvehicles navigate to a final position.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Definitions. As used in this description and the accompanying claims,the following terms shall have the meanings indicated, unless thecontext otherwise requires:

A “set” includes at least one member.

A “vehicle” is device for transporting a load over land or within astructure, wherein the load is selected from the group consisting ofgoods, a set of people, and combinations thereof.

A “personal vehicle” is a self-powered vehicle, equipped with a set ofsensors and a controller to manage its movement, and configured foroperation in collaboration with a user.

A “user” is an individual selected from the group consisting of a humanpedestrian and a human-controlled vehicle.

A “human-controlled vehicle” is a self-powered vehicle, having asteering system and a drive system, maneuverable over a path defined bydirect or indirect manual inputs of a human driver to the steeringsystem and the drive system. Optionally, the human driver may maneuverthe vehicle by remote control. A human-controlled vehicle is consideredto define a trajectory “autonomously.”

A “leader” is an individual, selected from the group consisting of userand a first etiquette-based vehicle, wherein the individual autonomouslydefines a trajectory of a second etiquette-based vehicle that is pairedwith the individual.

An “etiquette-based vehicle” is a personal vehicle configured to operatein any of a plurality of modes, such modes including a pair mode and asmart behavior mode.

A “pair mode” of an etiquette-based vehicle is a mode in which thevehicle is configured to follow, with hysteresis dynamics, a trajectoryof a user.

A “convoy” is a set of etiquette-based vehicles wherein each vehicle inthe set, other than any etiquette-based vehicle that is a leader andoperating in a smart behavior mode, is in pair mode with a distinctobject selected from the group consisting of (i) a leader and (ii)another vehicle of the set. In a convoy, etiquette-based vehicles of theset are configured to move in succession behind the leader, with thefirst etiquette-based vehicle of the set being selected from the groupconsisting of the leader and an etiquette-based vehicle locatedimmediately behind the leader, and successive etiquette-based vehiclesof the set being located thereafter.

A “convoy behavior” of a single etiquette-based vehicle in a convoy ofetiquette-based vehicles is that behavior required of the singleetiquette-based vehicle for its participation in the convoy.

The term “hysteresis dynamics” means operation of an etiquette-basedvehicle so as to exhibit latencies in following movements of the user ina manner consistent with ergonomic comfort of the user and pedestriancourtesy.

A set of etiquette-based vehicles are in a “line formation” when eachsuccessive vehicle is following, directly or indirectly, a trajectory ofa leader.

A “known path” is a trajectory over which a self-powered vehicle hasbeen trained to navigate autonomously.

A “training behavior” is a behavior of an etiquette-based vehicle inwhich it is configured to learn a trajectory as a known path.

A “smart behavior mode” of an etiquette-based vehicle is a mode,initiated by a trigger, in which the vehicle has exited from pair modeand executes an autonomous behavior. At a conclusion of the autonomousbehavior in the smart behavior mode, the vehicle is configured to pauseand to await an event in its environment that would determine its nextmode, and, in default of such an event, will re-enter pair mode.

A “park mode” of an etiquette-based vehicle is a mode in which thevehicle is at rest.

A “trigger” is a set of conditions in the environment of anetiquette-based vehicle (which environment may include behavior of theuser) under which the vehicle's set of sensors, in cooperation with itscontroller, causes the vehicle to exit from pair mode and to enter smartbehavior mode. In various embodiments, occurrence of a trigger issignaled by the etiquette-based vehicle to the user, for example, by aset of optical indicators (such as lighting) on the vehicle, or by anaudible indicator (which may optionally be on the vehicle), or by adashboard display on a device that is affixed to the vehicle or separatefrom the vehicle (such as via a smartphone), in communication with thevehicle, executing a program for controlling a set of behaviors of thevehicle.

“Pursuit Dynamics” are criteria for performance by an etiquette-basedvehicle in pair mode, with regard for the user's point of view, thebystander's perspective, and technical requirements for the vehicleitself. Such performance includes following position, start and stopdynamics, cornering and pivoting dynamics, and back-up dynamics.

A “sensor” in an etiquette-based vehicle is a device configured toprovide optical, radio, or other information about a set of objects inthe environment of the vehicle, wherein the information is provided as aset of electrical signals for processing by a controller to managemovement of the vehicle. In this context, a “sensor” includes amonocular camera, a stereoscopic camera, a radar imaging system, a LIDARsystem, etc.

A “dead zone” is an area defined, in relation to an etiquette-basedvehicle in pair mode with a user, within which the user is deemedstationary for purposes of following by the vehicle of movements of theuser.

Door-passing moments are five moments wherein an etiquette-basedvehicle, in accordance with embodiments of the present invention, isconfigured to perform a discrete action in the course of a door-passingsequence of actions. Door-passing moments collectively define a taxonomyof discrete actions that allow practical analysis and implementation ofthe vehicle's passage through a doorway, while each moment remains partof the larger door passage sequence. The five moments are as follows:

-   -   1. The following approach moment is the action wherein the        etiquette-based vehicle in pair mode is following a user up        until the moment when the user reaches a door. The vehicle        follows in heel following position, adjusting between the        dynamic following distances based on the user's speed and        tucking for obstacle avoidance as described below. The vehicle        detects that the user is stopping when the user's velocity dips        below 0.6 m/s, and is configured to reach its stop position at        450 mm directly behind the user when the user's velocity is less        than 0.2 m/s. In the following approach moment, in embodiments        of the present invention, there is provided an additional        obstacle detection requirement so that the vehicle provides to        the user enough space so that the user can comfortably open a        swing-in door.    -   2. The decoupling moment is the action wherein the vehicle        decouples from pair mode with the user, and initiates the        autonomous door behavior. This action, in embodiments of the        present invention, commences when the perceived door openness        angle, calculated from the vehicle's stopping position, as the        apparent angle between the door frame and the end of the open        door, reaches 60°.    -   3. The smarts moment is the action wherein the vehicle engages        in autonomous door passing behavior, including planning the path        for the vehicle's passage through the open doorway and        navigating through the doorway to the recoupling position        determined in the context of the specific door type.    -   4. The recoupling moment is the action wherein the vehicle        re-enters pair mode with the user. At the recoupling moment, the        vehicle is oriented in the most likely direction of the user's        gaze when the user closes the door. Gaze is defined as the        direction in which the front of the user's head is oriented. The        vehicle thereupon initiates pair mode with the use and engages        in following with hysteresis dynamics as described below.    -   5. The following walk-away moment is the action wherein the        vehicle transitions from a waiting position at the recoupling        moment to heel following position in pair mode. The vehicle        resumes following when the user moves beyond the track zone with        a velocity of at least 0.6 m/s. The vehicle is configured to        draft behind the user and then transition to heeling, as        described below.

Swing IN/Swing OUT refer to the door swinging direction of a side hingeddoor. The direction is determined relative to the direction of travel ofthe user. If the user must pull the door in order to open it, then it isa swing-IN door, whereas if the user must push the door in order to openit, then it is a swing-OUT door.

Embodiments of Dynamic Behavior by an Etiquette-Based Vehicle in PairMode.

In the definitions above, we have said that, in pair mode, anetiquette-based vehicle is configured to follow, with hysteresisdynamics, a trajectory of a user. Furthermore, “hysteresis dynamics”refers to operation of an etiquette-based vehicle so as to exhibitlatencies in following movements of the user in a manner consistent withergonomic comfort of the user and pedestrian courtesy. Since the userand the vehicle are both self-powered and constitute a dynamiccombination of components, the vehicle's controller is programmed tocause the vehicle to behave, i.e., to move, in accordance with theseprinciples, in a range of distinct dynamic contexts. The vehicle'sdynamic behavior in each of these contexts is programmed to conform to aset of models we have developed based on empirical measurements andrelated design considerations.

As to these design considerations, from the user's perspective, the pairmode establishes a sense of trust. The view of the vehicle in the user'speripheral vision, as well as the motion of tucking and untucking inresponse to obstacles, creates operator awareness. The heel followingposition draws inspiration from human-dog dyads, building upon analready familiar pairing. The vehicle in pair mode is programmed to beintuitive and to help to foster a bond between the user and the vehicle.From the bystander's perspective, the user and the vehicle are perceivedas a social unit with a magic factor. Heeling draws upon socialpedestrian dynamics to strengthen the pairing perception. The vehicle isvisible to oncoming pedestrians, increasing awareness. When thebystander comes closer to the vehicle, it tucks out of the way, modelinggood pedestrian etiquette. To minimize the width that the vehicle-userdyad takes up on a sidewalk, a narrow heel following position ispreferred. A decision on whether the vehicle's following position shoulddefault to one side, to the left or to the right of the user, is madewith considerations for social norms and takes into account regionaldifferences. By following social norms, the paired vehicle is perceivedas passive and polite, remaining in the slow lane and out of the way ofoncoming bystander traffic. The responsibility for vehicular movement isthus placed on the user as the leader of the vehicle, and the vehiclewill be cautiously following. Specifically, we address the followingcontexts for operation in pair mode: heel following position, obstacleavoidance, and draft following position.

Paired vehicle operation in heel following position. FIG. 1 is a diagramillustrating, in accordance with an embodiment of the present invention,from a top view, relative positions of a user 12 and an etiquette-basedvehicle, in pair mode, in successive speed-dependent heel-followingpositions 13A, 13B, and 13C relative to the user in accordance withembodiments of the present invention. We consider vehicle operation inthe context of three different user walking speeds: slow speed (0.6m/s), normal speed (1.2 m/s), and fast speed (2.1 m/s). At a slowwalking speed of 0.6 m/s in open spatial conditions, the vehicle'sposition 13A (determined at the center 131 of the vehicle) is 800 mm and21° relative to a fiducial reference at the rear of a centerline throughthe user at point 121. At normal walking speed of 1.2 m/s in openspatial conditions, the vehicle's position, measured at the center 132of the vehicle, is 1000 mm and 17° relative to the same fiducialreference point 121. Finally, at the fast speed of 2.1 m/s, thevehicle's position, measured at the center 133 of the vehicle, is 1450mm and 11° relative to the fiducial reference point 121. The vehicle'sfollowing position is intended to remain within a constant lateraloffset of 300 mm (from centerline of the user to centerline of thevehicle) at the three defined speeds. The angles defined above fallalong the 300 mm offset line and are meant to serve as guides. Walkingspeeds below 0.6 m/s trigger dynamic stopping behavior. The primary sideof the user on which the vehicle maintains following position should bebased on regional norms, so that, for example, in the U.S. the vehicleshould default to following on the right side of user, whereas in theU.K., Japan, and Australia, the vehicle should default to following onthe left side of user. This behavior should be defined at point ofregistration during onboarding based on location and should also be auser-controlled setting in a corresponding application for controllingvehicle behavior. If repeated failure occurs and the vehicle is unableto achieve its goal of following in heeling position, it should performfollowing in the draft following position, discussed below.

Paired vehicle operation in obstacle avoidance. FIG. 2 is a diagramillustrating, from a top view, relative positions of a user and anetiquette-based vehicle, in accordance with embodiments of the presentinvention, in pair mode, first (as in FIG. 1 ) in successivespeed-dependent heel-following positions 13A, 13B, and 13C relative tothe user 12, and second with an obstacle avoidance tuck (to avoid thepedestrian in position 21) showing the vehicle in successivespeed-dependent positions 23A, 23B, and 23C relative to the user 22.While performing obstacle avoidance, the paired vehicle executes a tuckbehavior—a transition defined by a cubic spline curve. Specifically,when the vehicle encounters an obstacle, such as pedestrian 21, it tucksbehind the user 22, transitioning to a 0° angle, measured from thecenterline of the user 22 at point 221 and maintaining the followingdistances, defined in the previous paragraph, dependent on speed. Thetuck behavior during the transition period can be represented by a cubicspline curve based on the following parameters: distance and anglebetween the vehicle and the user; velocity of the user; velocity of theobstacle; distance from the vehicle to the obstacle. The cubic splineequation is as follows:y=a(x−x ₁)³ +b(x−x ₁)²,where x₁, a, b are constants specific to the vehicle's velocity.

Paired vehicle operation in draft following position. Once the vehicleexecutes the tucking behavior, in accordance with embodiments of thepresent invention, it maintains a draft following position, in which thevehicle follows directly behind the user. At a slow walking speed of 0.6m/s in crowded spatial conditions, the vehicle's position is 800 mm and0° angle. At normal walking speed of 1.2 m/s in crowded spatialconditions, the vehicle's position is 1000 mm and 0° angle. At fastwalking speed of 2.1 m/s in crowded spatial conditions, the vehicle'sposition is 1450 mm and 0° angle. The vehicle is configured to return toheel following position under appropriate circumstances. In oneembodiment, the vehicle returns to heel following position after a setamount of time, i.e. 30 seconds, of uninterrupted detections at theuser's heeling side. In another embodiment, the vehicle is configuredwith a peeking feature under which it repetitively moves out to the sideby a threshold distance sufficient to support a view ahead whileremaining within the draft following threshold.

Dead zone and track zone parameters. In accordance with some embodimentsof the present invention, when an etiquette-based vehicle is in pairmode with a user, the vehicle disregards nominal movements by the user.However, it is not a simple matter to determine when a movement isnominal. To that end, we conducted studies to observe how much peoplemove while they perform simple everyday activities (such as looking forkeys to open a door, picking vegetables at a market, putting awaygroceries in a kitchen). These activities helped us define the zoneswithin which people move during an extended stop. Our data from suchstudies is summarized in FIG. 3 , which is a graphic mapping from ourstudies of motion of individuals performing tasks in a relativelystationary position.

We further analyzed the data summarized in FIG. 3 to determine geometricdistribution of data by percentiles. Table 1 shows our findings. Thedata in Table 1 was used to define what we call the “dead zone,” fromdata in the 75^(th) percentile, and the “track zone,” from data in the90^(th) percentile. In addition, we found that 99% of a person'smovement during stops has a velocity ≤0.6 m/s, which we define, inembodiments of the present invention, as the velocity threshold of theuser, above which the paired etiquette-based vehicle resumes followingthe user.

TABLE 1 % of X-distance Y-distance Angle Velocity Acceleration data(±mm) (±mm) (±º) (m/s) (m/s²) 10 29.8  17.5  2   0.014  0.56 20 53  36.4  4   0.022  0.92 25 66.1  48.6  5.5 0.027 1.1 30 81.9  61.8  7.30.032  1.28 40 130.5  95.5  10.9  0.044  1.69 50 177.3  148.2  15.3 0.061  2.21 60 237.8  198.4  20.4  0.084  2.87 70 300.1  258.3  26.3 0.116 3.8 75 363.9  298.7  30.7  0.141  4.41 80 425.8  344.3  34.9 0.174  5.19 90 528.5  496.6  56.6  0.284  7.83 99 833.9  858    158.8 0.622 20.2 

From the user's perspective, latency of the paired etiquette-basedvehicle's movements promotes simple decision-making as the user shiftsposition within the dead zone. The user is better able to navigatearound vehicle, knowing that the vehicle will stay in the position itwas in when the user first stopped. When the user moves beyond a definedarea deemed the dead zone, the vehicle turns in place to let the userknow that the vehicle is still with the user and keeps the user withinits field of view. This movement reassures the user that the vehicle isstill tracking the user, even though it does not react to the user'severy move. When the user moves beyond the track zone, the vehicleresumes following.

The bystander's perspective is also taken into consideration. Byminimizing the vehicle's movements over the timeframe of an extendedstop, the vehicle's footprint is reduced and is less likely to be in theway of other people. In addition, the vehicle appears more calm,courteous, and patient. By turning in place, the vehicle lets bystandersknow that it is paired with the user.

In accordance with embodiments of the present invention, the definitionsof “dead zone” and “track zone” give the etiquette-based vehicle in pairmode with the user an indication of how likely the user is to remainwithin a defined space or to continue walking. The vehicle tracks thetrajectory of the user's movements as the user is performing anactivity. By measuring the angle and distance to the user at any time,the vehicle is able to determine whether the user is staying or going.FIGS. 4A and 4B are diagrams of the dead zone and track zonerespectively in accordance with embodiments of the present invention.The “dead zone” is defined as a teardrop-shaped area with the followingdimensions measured from vehicle's center: 850 mm in the +x direction;60° angle at the vertex of the drop; 842 mm from the vertex to where thestraight edge intersects the circular edge; and a radius of 280 mm forthe circular portion of the drop. The “track zone” is defined as thearea bounded by a parabolic curve and cutoff angle with the followingdimensions measured from the vehicle's center: parabola with a vertex of(1100, 0); limits of ±600 mm in the ±y directions; 120° angle that openstowards the parabola; 660 mm from the vertex to the intersection of theparabola.

Pairing and Start. When the user pairs with an etiquette-based vehicle,our studies show that the user tends to stand about 680 mm in front ofthe vehicle. The vehicle is configured to give feedback to the userthrough lights and sound that the pairing has been successful, and itsmotion remains stable. Our analysis shows that 75% of the time the userwill be within the teardrop-shaped dead zone drawn from vehicle'scenter. In embodiments of the present invention, if the user is outsidethe dead zone when pairing, the vehicle is configured to rotate toorient and center itself with the user. In further embodiments of thepresent invention, the vehicle will begin following only once the userhas left the track zone. Upon pairing, in embodiments of the presentinvention, when the user is in the dead zone, the vehicle remainsstationary, giving the user feedback through the user interface of asuccessful pairing via lights and sound. If the user is outside the deadzone but within the track zone, however, the vehicle will adjust itsorientation so that the user is centered within the dead zone. Thevehicle is configured to remain stationary until the user leaves thetrack zone.

Once the user leaves the track zone with a velocity of at least about0.6 m/s, the vehicle is configured to begin following. If the user exitsthe track zone with a velocity less than about 0.6 m/s, the vehicle willdraft behind the user, as per design specification for stopping. As thevehicle adjusts its position and orientation to keep the user in itsfield of view, the dead zone and track zone shift with the vehicle.

Staying. When a person comes to a stop and performs a stationaryactivity for a few moments at a time, the person tends to move aroundwithin a limited area. Once in stop position, in accordance withembodiments of the present invention, the paired vehicle continues totrack the user while staying in the position it was in when the userfirst reached a stop. In a manner similar to that configured for pairingand start, if the user moves beyond the dead zone within the track zone,the vehicle turns in place to keep the user within its field of view.Only when the user moves beyond the track zone does the vehicle resumefollowing. In accordance with further embodiments of the presentinvention, the vehicle detects that the user is stopping and reachesstopping position when the user velocity is less than about 0.2 m/s. Invarious embodiments, the vehicle's stop position is 450 mm directlybehind the user. The vehicle will not move forward from this positionuntil the user moves beyond the track zone. When the user moves beyondthe dead zone into the track zone, the vehicle rotates in place tocenter itself with the user. In various embodiments, the vehicle resumesfollowing once the user moves beyond the track zone with a velocity ofat least 0.6 m/s. If the user exits the track zone with a velocity lessthan 0.6 m/s, the vehicle will draft behind the user, as per thevehicle's design specification for stopping. In the case wherein theuser halts too quickly and the vehicle fails to reach a stop position(450 mm directly behind), the vehicle remains where it is and turns toface the user. The vehicle will not move from this position until theuser moves beyond the track zone as projected from the vehicle's currentorientation.

Smart behavior mode and negotiation of doorways. We have defined “smartbehavior mode” of an etiquette-based vehicle as embracing a behavior,initiated by a trigger, in which the vehicle has exited from pair modeand executes an autonomous behavior. At a conclusion of the autonomousbehavior in the smart behavior mode, the vehicle is configured to pauseand to await an event in its environment that would determine its nextmode, and, in default of such an event, will re-enter pair mode. Intypical embodiments of the present invention, the vehicle performsprocesses including temporarily unpairing from the user; autonomouslydriving through a doorway while the door is held open by the user thevehicle has been following; driving to a position on the other side ofthe doorway and waiting for the user to walk through the door;re-entering a pair relationship with, and following, the user.

The design of the autonomous behavior, in accordance with embodiments ofthe present invention, is informed by field observations and motioncapture studies. Six hinged, swinging door types were studied:Left-swing IN, Left-swing OUT, Right-swing IN, Right-swing OUT, DoubleDoors-swing IN, and Double Doors-swing OUT. Direction of hinge and swingare defined in relation to the user's direction of travel. Doors that donot hang in a frame, hinge from one side, and swing open along theirvertical axis to a minimum of 90 degrees are not contemplated in thisspecification, although, some of the design principles articulatedherein may apply. Furthermore, the research study design and designspecifications are constrained to approaching a door head-on,perpendicular to the door face. Nevertheless, use of a human-followingrobot provides an advantage to leverage the human ability to open doorsand to hold them open.

Movement of a machine through a doorway with a user, while the userholds the door open, involves a more sophisticated behavioral model ofthe machine's motion than a model wherein the machine simply staysbehind the user and matches the user's forward motion. Configuringautonomous behaviors in this context allows the vehicle to help the userto tackle challenging situations. The Smart Behaviors for Doors hereinare designed to make the user experience of walking through a door withthe robot feel similar to holding a door open and walking through thedoorway with a person or other intelligent companion that understandsetiquette associated with door opening, holding, and passing throughdoorways.

Our field observations and motion capture studies focused on peoplemoving in pairs or with a rolling object through the six types of doorslisted above. A reconfigurable door set was built specifically formotion capture. Twenty-four outside participants were hired with theinstructions to walk through the doors together with one person openingand holding the door and another following through. The participantswere not told about the reason for or use of this data, so as not tobias their behavior. They were asked variously, as leaders andfollowers, to walk through the six types of doors. We collected 2,240passes of people walking through the six types of doors.

Our analysis framework was driven by the following set of questions:

-   -   Where does a person typically stop to open a door?    -   Where does someone or something following a person typically        stop when waiting for a door to be opened?    -   How much does a person move when opening an inward-swinging        door?    -   What is the openness of the door from the perspective of the        follower when approaching the door?    -   At what moment does the follower decouple from the leader and        initiate their walk through the door?    -   How much time passes from when the leader starts opening a door        to when the follower initiates their walk through the door?    -   While passing through an inward-swinging door, how much space        does a person leave between them and a person holding the door        for them?    -   Where does someone or something following a person typically        wait to recouple with the leader after passing through a door?    -   Where does a person typically stop to close a door?    -   Does the leader gaze indicate the recoupling moment before        walking away together? For how long does the gaze overlap last?    -   How much time elapses between the decoupling moment at door        opening to the recoupling moment at door closing?

Insights that drove the data-driven Doors Behaviors Design focused onleader and follower spatial dynamics, timing, gaze analysis, andperceived openness metrics. From the six door types which were studied,no significant differences were observed between single doors and doubledoors. For this reason, the resulting door specifications focus on fourdoor types: Left-swing IN, Left-swing OUT, Right-swing IN, andRight-swing OUT. The data from double doors were divided into doubledoors left and double doors right, and grouped with the insights forsingle left and single right doors. Qualitative design considerationsand technical specifications for the etiquette-based vehicle guided themethodology for how the insights were applied as final design decisions.

LEFT-SWING IN. Relative to the direction of travel of the user, aLeft-swing IN door is mounted on the left side of a frame and swingsinward towards the user. The total time for a pair passing through aLeft-swing IN door is 7.10 seconds.

Following approach moment. FIG. 5 is a diagram illustrating, from a topview, relative positions of a user 51 and an etiquette-based vehicle, inpair mode, in successive positions 52 and 53, as the vehicle slows downin a following approach moment (as defined above) in relation to door54, in accordance with an embodiment of the present invention. Ourstudies show that when a person approaches a Left-swing IN door, asillustrated in FIG. 5 , there is a significant variance of the leader'sposition towards the door handle location. The leader reaches for thehandle at 400 mm from the door center line towards the door handle and821 mm orthogonally from the door. Time is defined as (t=0) in the doorpassage sequence at the moment the leader reaches for the handle. Thestopping position of the user reflects these observations. Our studiesalso show that at the moment that the leader reaches for the handle of aLeft-swing IN door, the follower's position is 383 mm from the doorcenter line and 1436 mm orthogonally from the door. The follower isclosely aligned behind the leader. Our understanding of how peopleapproach a Left-swing IN door, leads us to specify that the stoppingposition of an etiquette-based vehicle should be 1500 mm orthogonallyfrom the door, aligned behind the user in accordance with the StopSpecifications.

Decoupling moment. In order to determine when a person opens and holds adoor open for a robot to move through autonomously, we observed themoment at which the follower decouples from the leader. The decouplingscenario is defined by the perceived openness angle, the angle from thefollower's position measured to the door frame and the door end. InLeft-swing IN doors, decoupling happens when the follower initiateswalking through a door that is being held open by the leader. Theperceived door openness angle at decoupling is 60°, and the actual doorangle is 68°. The width of the door opening from the door frame to thedoor end is 1000 mm. Time when decoupling occurs is at t=1.32 seconds.To understand the space that the user occupies while moving to open andhold open a Left-swing IN door, the user's movement is visualized as theopening movement zone. The opening movement zone is graphicallyrepresented as item 64 in FIG. 6 , which is a diagram illustrating, froma top view, relative positions of a user 61 and an etiquette-basedvehicle 62, in pair mode, in the decoupling moment (as defined above) inrelation to a door 63, in accordance with an embodiment of the presentinvention. The opening movement zone 64 is defined by four extentvalues: orthogonally from the door: min 390 mm and max 1216 mm; fromdoor center line: min 470 mm to the right and max 277 mm to the left, inthe direction of travel.

Smarts moment. FIG. 7 is a diagram illustrating, from a top view,relative positions of a user 71 and an etiquette-based vehicle 72, inthe smarts moment (as defined above) in relation to a door, as thevehicle 72 navigates autonomously over path 73 through the doorway inaccordance with embodiments of the present invention.

Our research studies revealed that the follower holds a distance of 882mm of personal space when passing the leader. Based on the recordedleader and follower positions, our recommended distance is given inrelation to the door center line. Beginning at the moment of decoupling,at initiation of the smarts moment, the etiquette-based vehicleinitiates the Left-swing IN autonomous path 73 through the door. In thisbehavior, the vehicle first makes a wide turn with a radius of 650 mmperpendicular to the door center line, to the right in the direction oftravel, and then veers back to align itself with the door centerline asit passes through the threshold. As observed in the research studies,the follower's waiting position on the other side of the door was 973 mmfrom the door center line and 707 mm orthogonally from the door. Thisobservation is used to configure the waiting position of theetiquette-based vehicle on the other side of the door: 1000 mmperpendicular to the door center line and 500 mm orthogonally from thedoor. The smarts moment starts at decoupling (t=1.32 seconds) and endsat recoupling (t=4.27 seconds), a total of 2.95 seconds for theautonomous passage through the door.

Recoupling moment. FIG. 8 is a diagram illustrating, from a top view, inaccordance with an embodiment of the present invention, relativepositions of a user 81 and an etiquette-based vehicle 82, in therecoupling moment (as defined above) in relation to a door, after thevehicle 82 has navigated autonomously through the doorway, when the usercloses the door and the vehicle is oriented in the most likely directionof the user's gaze. The recoupling moment begins when the user's gazeoverlaps that of the etiquette-based vehicle in acknowledgement ofrejoining and readiness to continue walking. The vehicle's position atthe recoupling moment is determined by the observation that 74% of thetime a person looks towards the door opening while closing a Left-swingIN door. The recoupling moment occurs at t=4.27 seconds in the doorpassage sequence. The length of time in which the leader looks in thefollower's direction is about 0.28 seconds. The vehicle is expected tore-enter pair mode with the user within 0.28 seconds.

Following walk-away moment. FIG. 9 is a diagram illustrating, from a topview, relative positions of a user 91 and an etiquette-based vehicle 92,in the following walk-away moment (as defined above), in accordance withan embodiment of the present invention, after the recoupling moment andvehicle 92 has re-entered pair mode with the user. When the user movesbeyond the track zone with a velocity of at least 0.6 m/s, the vehicleis configured to operate in draft following position, centering itselfdirectly behind the user, and then to transition to heel followingposition. The time duration from the recoupling moment to the followingwalk-away moment is 2.55 seconds. Following walk-away moment occurs att=7.10 seconds in the door passage sequence.

RIGHT-SWING OUT. Relative to the direction of travel of the user, aRight-swing OUT door is mounted on the right side of a frame and swingsoutward away from the user. The total time for a pair passing through aRight-swing OUT door is 7.09 seconds. Passing through a Right-swing OUTdoor involves the same five door-passing moments as just described for aLeft-swing IN door. Compared to the situation for a Left-swing IN door,the distances and timings are analogous, albeit not identical, since thegeometries in each case are differing.

FIGS. 10 and 11 are front and rear views respectively of a smart-followmodule fitted with a set of sensors and a controller configured tomanage movement of an autonomous host self-powered vehicle, inaccordance with another embodiment of the present invention.

Smart behavior mode and travelling along known paths. Non-commercialowners of etiquette-based vehicles, whether living in urban apartments,suburbs, or planned developments, are using the vehicles primarilyoutside of the home. After taking the vehicle out, the owners need tostore the vehicle in a place, which is typically narrow, tucked away, ordifficult to get the vehicle to access in a following mode. Similarly,commercial owners of etiquette-based vehicles have a need to store thevehicles in methods customized to their business venues, and may alsohave fleets of vehicles that need to be navigated. Teaching a vehicle totravel along a known path, allows the vehicle's owner to optimizeaccessibility of the vehicle in difficult-to-navigate spaces, renderingunnecessary the burden of otherwise leading the vehicle into suchspaces.

A known path has been defined above as a trajectory over which aself-powered vehicle has been trained to navigate autonomously. Anetiquette-based vehicle using a training behavior learns, from a user, atrajectory as a known path as shown in FIGS. 12-14 .

FIG. 12 is a diagram illustrating, from a top view, relative positionsof a user 1201 and an etiquette-based vehicle 1202, at the start of atraining behavior (as defined above), in accordance with an embodimentof the present invention, before the user 1201 has shown theetiquette-based vehicle 1202 the known path. In some embodiments of theinvention, when the user 1201 starts the training behavior, the vehicle1202 will run a 360 degree scan of the environment around the vehicle'scenter 1204. In one embodiment of the invention, the vehicle 1202creates a map of its surroundings from data collected in the 360 degreescan. After the scan, the vehicle 1202 will enter pair mode and with theuser. As shown in FIG. 12 , the vehicle has performed a scan and rotatedto be behind the user 1201 and ready to follow such user. Since the user1201 and the vehicle 1202 are larger than a point in space, in someembodiments, the vehicle will follow a trajectory of the user's center1203 with the vehicles center 1204. In other embodiments, the vehicle1202 follows the user's heel. In further embodiments of the invention,the vehicle may follow a wearable attachment.

In various embodiments of the invention the user can begin and end atraining behavior of the vehicle through an app, a button on thevehicle, a somatic signal, a verbal signal, or other methods known toone having ordinary skill in the art.

FIG. 13 is a diagram illustrating, from a top view, in accordance withan embodiment of the present invention, an etiquette-based vehiclemoving along a trajectory 1320 from a start point 1304 to an end point1314, following a user 1301, 1311. In FIG. 12 , the user 1201 enteredthe training behavior, and the vehicle 1202 performed a scan. In FIG. 13, the user 1301 moves along the trajectory 1320, which is to be taughtto the vehicle 1304. In various embodiments, the vehicles center 1304follows different points 1303 of the user 1301. When the user 1311 hasbrought the vehicle 1312 to an endpoint of the trajectory 1320, the user1311 may end the training behavior such that the vehicle will rememberthe end point 1314.

In some embodiments of the invention, the speed at which the user 1301moves, and thus the corresponding speed at which the etiquette-basedvehicle 1302 moves, will be the speed that the etiquette-based vehiclemoves when it travels over the known path in the future. The trajectoryfollows the center of the user 1303 and the center of the vehicle 1304.

FIG. 14 is a diagram illustrating, from a top view, an etiquette-basedvehicle and a user, at the end of a training behavior, in accordancewith an embodiment of the present invention, wherein the vehicle 1412followed the user 1411. In accordance with embodiments of the invention,when the user 1411 ends the training behavior, the vehicle marks itslocation 1414 as the end location of the path. In some embodiment of theinvention, the vehicle remembers an area in which the training behaviorwas ended, and thus can recreate the known path from any point withinthat area. In some embodiments of the invention, when the training modeis complete, the user, or other operator, stores the known path with aname. In derivative embodiments, each direction of the path (i.e. startto end or end to start) may have its own name. This is advantageous asthe user can select the path a plurality of known paths. The paths maybe displayed on an app or other menu and having selections which directthe vehicle to move along the chosen path. In some embodiments of theinvention, the vehicle remembers its orientation at the end of the path,and will recreate said orientation after travelling along the known pathin the future. Similarly, in some embodiments, after completing thetraining behavior, the vehicle will rotate 540 degrees. 540 degreesallows for the vehicle to make a 360 degree scan and also orientateitself to return along the known path.

In some embodiments, to trigger the travel along the known path tostart, the robot must be within a radius of the path start point, forexample, one meter. In other embodiments, the travel along the knownpath is triggered by entering a specific mode. For example, in oneembodiment, when the vehicle leaves park mode and is in a specific area,for example, its parking spot, the vehicle will automatically travelalong a known path to be ready for function.

In some embodiments of the invention, the etiquette-based vehicle isable to adapt the known path to a smooth trajectory. In such embodimentsthe vehicle can interpolate a smooth trajectory from the path walked bythe user to determine the known path. In further embodiments, thevehicle can use sensors to determine whether obstacles are in the knownpath. In such embodiments, the vehicle can stop and wait for theobstacles to clear, send an alarm, and find pathways around theobstacles.

FIG. 15 is a diagram illustrating, from a top view, an etiquette-basedvehicle 1501 exiting smart behavior mode and entering park mode aftertravelling along a known path, in accordance with an embodiment of thepresent invention. In the embodiment shown in FIG. 15 , the vehicle 1512traverses the trajectory from the starting location 1514 to an endinglocation 1504. In some embodiments, the vehicle 1501 turns itself, aboutits axis, to face the direction it will move if reversing along saidknown path. In situations where the known path returns the vehicle to astorage spot, this is advantageous as the vehicle will more quickly beable to emerge from said storage spot when asked.

FIG. 16 is a diagram illustrating, from a top view, in accordance withan embodiment of the present invention, an etiquette-based vehicle 1612in smart behavior mode following a known path 1622 (as defined above). Auser, which follows a different path 1623, is shown to portray thevehicle moves without regard to the user. Furthermore, the embodiment ofFIG. 16 shows that the vehicle is still in smart-behavior mode as it isable to move autonomously, however, it is not in pair mode, as thevehicle does not follow the user. Additionally, FIG. 16 illustrates thatthe end location 1604 after travelling along a known path is, in someembodiments, a location 1642, here shown as a circle. Since the location1642 is a larger than a point, the vehicle is better able to navigateobstacles or just ignore obstacles that are slightly obscuring thetrajectory 1622 or location at the end of the known path. Similarly, thestarting location 1614 is defined by a circle 1641. In this embodiment,the user 1611 may start the travel along the known path when the vehicle1612 has entered anywhere in the radius 1641. This alleviates the needfor precision which may be a hassle. When the vehicle 1601 has parked,in some embodiments, the vehicle performs a scan to determine the areaaround such vehicle.

Embodiments of Dynamic Behavior by Etiquette-Based Vehicles in a Convoy.

In the definitions above, we have said that, in a convoy each of a setof etiquette-based vehicles are in in pair mode with a distinct objectselected from the group consisting of (i) a leader and (ii) anothervehicle of the set. In a convoy, etiquette-based vehicles of the set areconfigured to move in succession behind the leader, with the firstetiquette-based vehicle of the set located immediately behind theleader, and successive etiquette-based vehicles of the set being locatedthereafter.

In some embodiments, the convoy behavior of the etiquette-based vehicleshas three stages. The first stage, shown in FIG. 18 , involves gettingin line. The vehicles are originally spaced and will fall in line as theuser 1801 executes traversal of a trajectory. The second stage, shown inFIG. 19 , is following in line. The vehicles travel, with hysteresisdynamics, successively in line behind the user 1905. The third stage,shown in FIG. 20 , is getting out of line, during which each of thevehicles exits pair mode and executes a smart behavior to return to agrouping different from a line formation.

Furthermore, in some embodiments of the invention, each vehicle in theconvoy moves with hysteresis dynamics. As defined above, “hysteresisdynamics” refers to operation of an etiquette-based vehicle so as toexhibit latencies in following movements of the user in a mannerconsistent with ergonomic comfort of the user and pedestrian courtesy.Since the user and the paired vehicle are self-powered and constitute adynamic combination of components, the vehicle's controller isprogrammed to cause the vehicle to move in a range of distinct dynamiccontexts.

FIG. 17 is a diagram illustrating a group of etiquette-based vehicles1702, 1703, 1704, 1705, 1706 forming a convoy in accordance with anembodiment of the present invention. In some embodiments, the vehiclesthat form the convoy start within a specific radius 1741, which is, forexample, 1.9 meters. In further embodiments, each robot that the userwould like to place in the convoy must be within 1 meter of the nearestrobot also selected for the convoy. In one embodiment of the invention,after initialization of the convoy behavior, each robot enters pair modewith the vehicle it will be following. In some embodiments, a user 1701may select a vehicle 1702 with which to enter convoy behavior, and suchvehicle would pair with the user 1701, and thus be the first vehiclebehind the user. The vehicle 1702 would then find the other vehicleswithin a proximity and communicate to those vehicles to enter convoybehavior as well. Then, a second vehicle 1704 pairs with the firstvehicle, and a third vehicle 1706 pairs with the second vehicle. Thisprocess is continued until each vehicle has been paired. In someembodiments, the order of the vehicles in the convoy is determined byproximity to the user 1701. In other embodiments the order is randomlydetermined.

FIG. 18 is a diagram illustrating, from a top view, in accordance withan embodiment of the present invention, a plurality of etiquette-basedvehicles in the first stage of convoy behavior (as defined above)following a user 1801. In one embodiment of the invention, once theleader, in FIG. 18 user 1801, begins walking, a first vehicle willfollow the leader, a successor vehicle will follow the first vehicle,and so on, so that the vehicles enter a line formation. In oneembodiment, the first vehicle 1802 follows the user after 1.8 seconds.In this embodiment, each successive etiquette-based vehicle (which wesometimes call a “robot”) 1803, 1804, 1805, and then 1806, will be inthe final following position 1.3 seconds after the previous robot. Thatis, robot 1803 will enter the line formation 3.1 seconds after theleader began walking, robot 1804 will enter the line formation 4.4seconds after the leader began walking, and so on until each vehicle hasentered the line formation. In different embodiments, the time to enterthe line formation depends on the user's walking speed and otherfactors. In the embodiment shown in FIG. 18 , the order was chosen byproximity to the first following position. Once each vehicle had enteredthe convoy, vehicle 1802 is in place 1812, vehicle 1803 is in place1813, vehicle 1804 is in place 1814, vehicle 1805 is in place 1815, andvehicle 1806 is in place 1816.

FIG. 19 is a diagram illustrating, from a top view, in accordance withan embodiment of the present invention, a plurality of etiquette-basedvehicles in convoy behavior (as defined above) following a user withhysteresis dynamics. Hysteresis dynamics within the convoy aids inportraying the convoy as a plurality of individual vehicles workingtogether instead of a robotic cluster of vehicles. This human feelprovides a sense of comfort to the user and viewers alike. In someembodiments, following with hysteresis dynamics is achieved by thevehicles' following at different distances based on the speed of theobject they are following. In one embodiment, the vehicle follows 600 mmbehind an object moving 0.6 m/s; 800 mm behind an object moving at 1.2m/s; and 1250 mm behind an object moving at 2.1 m/s. Such a vehiclestays 450 mm behind a stopped vehicle. Each distance is shown in FIG. 19by the groups of vehicles 1901, 1902, and 1903. The first vehicle ineach group represented by 1901 a, 1902 a, and 1903 a is a set distancebehind the user 1905 depending on the speed of the user. These followingdistances mimic typical human behavior when following another human. Thefollowing distance for group 1901 is shown in 1910, the followingdistance for group 1902 is shown in 1902, and the following distance forgroup 1903 is shown in 1930.

To overcome obstacles in the path, a vehicle's path in the convoy maydeviate from the direct trajectory of the object it is following. Insome embodiments, the vehicle may be allowed a certain amount of gracewithin which to leave the path of the object it is following, forexample, 240 mm.

FIG. 20 is a diagram illustrating, from a top view, in accordance withan embodiment of the present invention, a plurality of etiquette-basedvehicles as convoy behavior is toggled off and the etiquette-basedvehicles get out of line. In various embodiments, convoy behavior isturned off through an app, a somatic signal, a verbal signal, orpressing a button on the vehicle. In some embodiments, when convoybehavior is terminated, the etiquette-based vehicles 2002, 2003, 2004,2005, 2006 use a smart behavior to enter into a cluster near whereconvoy mode has been terminated. In these embodiments, when convoybehavior of a vehicle is terminated, the behavior includes atransitional aspect during which the vehicle engages in leaving the lineformation to form, with other vehicles formerly in the convoy, acluster. In one embodiment, once the vehicles are all in the cluster,the clustered vehicles enter park mode. As shown in FIG. 20 , vehicle2002 was not required to move to form the cluster. In similarembodiments, other vehicles, for example, including vehicle 2003, arenot required to move to form a cluster. In one embodiment, if thevehicles are in park mode when convoy behavior is terminated (forexample, because the user stopped moving and thereafter ended convoybehavior), they will briefly exit park mode and perform a smart behaviorto move into a radius and then re-enter park mode. In other embodiments,when convoy behavior is terminated, the vehicles immediately enter parkmode. In further embodiments, when convoy behavior is terminated, thevehicles will travel along a known path. In some embodiments, thevehicles organize themselves such that each vehicle is in a resting spotat a similar time. Therefore, the vehicles in the back of the line wouldbe in the back of the cluster and vice versa.

Sensor System Embodiments for Use by an Etiquette-Based Vehicle in PairMode.

Because, in pair mode, an etiquette-based vehicle is configured tofollow, with hysteresis dynamics, a trajectory of a user, it isessential for the etiquette-based vehicle to obtain information as tothe user's position over time, and, also usefully, the user'sorientation over time. For this purpose, the etiquette-based vehicle isprovided with a set of sensors.

Optical imaging systems using electronic image sensors provide highresolution color and intensity information representing a 3-D sceneprojected onto the image plane. The image representation is typicallydivided into picture elements (pixels) arranged as a rectangular array.Modern solid-state sensors can supply tens of megapixels resolutions.Typical consumer CMOS sensors range from 5-20 megapixels, and industrialgrade sensors range from 1-10 megapixels. Many applications require anunderstanding of the range to each element in the scene, whichrepresents the third dimension of the image, the “depth.”

There are two commonly used technologies for achieving depth informationusing optical imaging, Time-of-Flight (TOF) imaging and stereoscopicranging. In a system using TOF imaging, a bright infra-red light isflashed from a location proximate to the image sensor along the sameviewing axis to the scene. Each pixel in the sensor measures the timefrom the activation of the flash until the reflected light is receivedat the pixel, so as to provide an estimate of the distance to theilluminated portion of the scene. There are numerous disadvantages tothis type of system, including high power requirements, low resolution,and range.

In a stereoscopic ranging system, two identical cameras are arranged ina fixed and stable relationship. Two images are captured simultaneously,and the resulting images are compared pixel by pixel to provide a 2-Dimage representation along with a range estimate for each pixeldetermined by measuring the pixel disparity between the two cameras forfeatures in the scenes. Stereoscopic cameras can use any image sensor,so the sensors selected for the application can be selected for adesired image quality. Because stereoscopic cameras do not require theflashing of a bright light, they consume less power than TOF cameras.Also, the range and precision can be optimized by selecting theresolution of the imager and the distance between them in the apparatus.

Monocular imaging systems cannot by themselves provide range data, butif the target is identifiable, they can provide bearing information.Optimizing the performance of a ranging imager generally requires itsFOV to be limited to 90 degrees or less. In situations where the leadermoves to the side of the FOV of the 3D ranging system, the angularbearing can still be determined in the wide FOV monocular camera toallow the vehicle to be rotated to the right heading to recapture theleader in the 3D sensor FOV.

In a first embodiment of a set of sensors for use with anetiquette-based vehicle in pair mode, we combine the use of astereoscopic ranging optical imaging system with a wide FOV monocularimaging system. Both sensors supply information to a tracking systemused by the vehicle to determine the location of the user being followedby the vehicle. The tracking system monitors the point information alongwith the image information to calculate the location of the user withrespect to the vehicle. In the event that one sensor or the other iscompromised by some external factor, such as sunlight, the other sensorcan still provide adequate information to maintain operation until thedisturbance is removed. In situations where the user moves to the sideof the FOV of the 3D ranging system, the angular bearing can still bedetermined in the wide FOV monocular camera to allow the vehicle to berotated to the right heading to recapture the leader in the 3D sensorFOV.

In a second embodiment of a set of sensors for use with anetiquette-based vehicle in pair mode, we combine 3D optical imaging with3D radio imaging to provide user tracking information to the vehiclewhen in pair mode with the user. An optical imaging system employingmachine vision techniques and a high-resolution multi-antenna radarimaging system to provide robust 3D point clouds under changingenvironmental conditions. The system includes a wide FOV monocular colorcamera, and a 4D (x,y,z,v) radar ranging sensor. The camera and radarsystems are placed near each other on the vehicle and have overlappingfields-of-view (FOV).

Radar systems employ RF signal emissions and reflections to identify therange to a target. Multiple antenna systems can provide range to targetsalong with X,Y or elevation and azimuth information. Modern solid stateradar systems can provide high resolution imaging providing x,y,z andvelocity information for each voxel sensed. Radar systems, however,cannot operate at the same speed as optical systems, do not yet have thesame high resolution, and do not provide any information about color inthe scene.

In second embodiment, we combine the use of a stereoscopic rangingoptical imaging system with a high-resolution radar system. Both sensorssupply information to the tracking system. The tracking system monitorsthe point information along with the image information to calculate thelocation of the leader with respect to the vehicle. In the event thatone sensor or the other is compromised by some external factor, such assunlight, the other sensor can still provide adequate information tomaintain operation until the disturbance is removed.

In a third embodiment of a set of sensors for use with anetiquette-based vehicle in pair mode, we combine a wide FOV monocularoptical imaging system with a high-resolution radar system. Both sensorssupply information to the tracking system. The tracking system monitorsthe point information along with the image information to calculate thelocation of the user with respect to the vehicle. In the event that onesensor or the other is compromised by some external factor, such assunlight, the other sensor can still provide adequate information tomaintain operation until the disturbance is removed. In situations wherethe user moves to the side of the FOV of the 3D ranging system, theangular bearing can still be determined in the wide FOV monocular camerato allow the vehicle to be rotated to the right heading to recapture theleader in the 3D sensor FOV.

In a fourth embodiment of a set of sensors for use with anetiquette-based vehicle in pair mode, we combine 3D optical imaging with3D radio imaging and wide FOV monocular imaging to provide leadertracking information for a following vehicle. Specifically, the set ofsensors includes an optical imaging system employing stereoscopicranging techniques and a high-resolution multi-antenna radar imagingsystem to provide robust 3D point clouds under changing environmentalconditions. The set of sensors further includes a stereoscopic rangingcamera system and a 4D (x,y,z,v) radar ranging sensor. The stereo andradar systems are placed near each other on the vehicle and have similarand overlapping fields-of-view (FOV). An additional wide FOV monocularimaging camera is included to provide bearing information when theleader steps outside the FOV of the 3D sensors.

In the event that one sensor or the other is compromised by someexternal factor, such as sunlight, another sensor can still provideadequate information to maintain operation until the disturbance isremoved. In situations where the user moves to the side of the FOV ofthe 3D ranging systems, the angular bearing can still be determined inthe wide FOV monocular camera to allow the vehicle to be rotated to theright heading to recapture the user in the 3D sensor FOV.

FIGS. 10 and 11 are front and rear views respectively of a smart-followmodule fitted with a set of sensors and a controller configured tomanage movement of an autonomous host self-powered vehicle, inaccordance with another embodiment of the present invention. When thesmart-follow module is implemented in the host vehicle, the host vehiclethen operates as an etiquette-based vehicle, in the manner described inconnection with the previous figures, and thus is configured foroperation in collaboration with a user as previously described.

Using video and radar sensing, the module can identify a target user andprovide range and heading information to the host vehicle for trackingand following the user. As described with respect to other embodiments,the module provides for intelligent trajectory planning and behaviorcontrol in an intuitive context that is comfortable for pedestrians. Themodule is equipped with an ethernet connection for integrating with thehost vehicle system. An API allows for setting of module or vehiclespecific parameters, modes of operation, data rates, and data formats.The API also allows for operational control of the system, setting upWiFi access, and updating system software. All measurement and log dataare communicated over this port when operating, as real time data. Videoand sensor image streaming is available (at reduced rates) over theseparate WiFi connection. In use, of the module, the vehicle hostprovides the module with the necessary operational settings, such as:height above the ground plane, inclination angle, maximum target range,target type (person, machine), data rate (10-30 Hz), and data format(range, heading angle units). In various embodiments, the user selectsthe operating mode: pure follow, heeling, custom offsets. After thesystem is started, the user presses the “pair” button the vehicle anduser are in position. The module then begins processing the user'sheading information and makes the information available to the hostusing the specified protocol. The host then operates the vehicle basedon the information coming from the module. If the module loses pairing,a lost message is sent to the host. When the task is complete, theoperator can push the pair button to unpair, or the host can send a stopcommand to the module.

The embodiments of the invention described above are intended to bemerely exemplary; numerous variations and modifications will be apparentto those skilled in the art. All such variations and modifications areintended to be within the scope of the present invention as defined inany appended claims.

What is claimed is:
 1. An etiquette-based vehicle comprising: amechanical drive system to cause movement of the vehicle; a set ofsensors; and a controller, coupled to the mechanical drive system, tomanage movement of the vehicle; wherein the controller is configured tocause the vehicle to exhibit convoy behavior so as to participate, witha set of similar etiquette-based vehicles, in a convoy beingestablished, the convoy being established having a leader, the convoybehavior including: getting in a line formation and following in theline formation, wherein each etiquette-based vehicle of the convoy beingestablished, other than any etiquette-based vehicle that is the leader(i) enters a pair mode with a distinct object, the object selected fromthe group consisting of the leader and another vehicle of the pluralityof etiquette-based vehicles and (ii) successively joins the convoy andfollows the leader, directly or indirectly, with hysteresis dynamics, asthe leader executes traversal of a trajectory.
 2. The etiquette-basedvehicle according to claim 1, wherein the convoy behavior furtherincludes a convoy exit process, during which each such etiquette-basedvehicle exits pair mode and enters a smart behavior mode, wherein eachsuch etiquette-based vehicle autonomously moves out of the lineformation.
 3. The etiquette-based vehicle according to claim 2, whereinthe convoy exit process further includes forming a cluster by theplurality of vehicles, and upon formation of the cluster entering a parkmode by each of the plurality of vehicles.
 4. The etiquette-basedvehicle according to claim 2, wherein the convoy exit process furtherincludes travelling along a known path by each of the plurality ofvehicles.
 5. The etiquette-based vehicle according to claim 1, wherein,in getting in the line formation: an etiquette-based vehicle, of theconvoy being established and located closest to the leader, is caused toenter pair mode with the leader; and each successive remaining vehicle,of the convoy being established and located next closest to the leader,is caused to enter pair mode with a corresponding vehicle havingpreviously entered pair mode.
 6. The etiquette-based vehicle accordingto claim 1, wherein the leader is an etiquette-based vehicle that hasbeen trained to travel autonomously along a known path.
 7. Theetiquette-based vehicle according to claim 1, wherein the set ofvehicles occupy arbitrary locations at the time they start the convoybehavior.
 8. The etiquette-based vehicle according to claim 1, whereinan order in which the set of etiquette-based vehicles follow each otheris determined by their proximity to the leader at the time they startthe convoy behavior.
 9. The etiquette-based vehicle according to claim1, wherein the vehicle follows about 600 mm behind an object moving atabout 0.6 m/s.
 10. The etiquette-based vehicle according to claim 1,wherein the vehicle follows about 800 mm behind an object moving atabout 1.2 m/s.
 11. The etiquette-based vehicle according to claim 1,wherein the vehicle follows about 1250 mm behind an object moving atabout 2.1 m/s.
 12. The etiquette-based vehicle according to claim 1,wherein the vehicle stays about 450 mm behind a stopped vehicle.
 13. Theetiquette-based vehicle according to claim 1, wherein the vehicledeviates from the direct trajectory of the object it is following. 14.The etiquette-based vehicle according to claim 13, wherein the vehicledeviates from the direct trajectory of the object by a grace amount. 15.The etiquette-based vehicle according to claim 14, wherein the graceamount is about 240 mm.